Bayesian models of cognition

• may require specifications of subjective beliefs

new experimental measure

• binned histogram task (Kao et al., 2014)

Bayesian data analysis

• great tool to specify and test ideas about how unobservables (like beliefs) relate to each other and how they generate observable data

relevance

• belief + desire + rationality \(\Rightarrow\) action

problem

• beliefs are latent, only action is directly observable

context

Joe is an adult male from Texas. He went to a BBQ party yesterday.

real-world frequencies

• may not exist
• subjects may not be aware

experimental measures

• give-number task & inference of parameterized distribution
  • e.g., Manski (2004); Tauber & Steyvers (2013)
  • scoring rules (Savage, 1971)
• iterated learning tasks (Lewandowsky, Griffiths, & Kalish, 2009)
• binned histogram task (Kao et al., 2014)

• 50 participants recruited via MTurk
• each saw every condition of every task
• 8 items (from previous research)
• 3 task types:
  • BH: binned histogram
  • GAN: give-a-number
  • PC: paired comparison

dummy

normalize slider rating by subject \(\Rightarrow\) average over subjects \(\Rightarrow\) BH task averages

dummy

dummy

goal: scrutinize BH task

• do BH task averages approximate the central tendency of beliefs in the population?

dummy

approach: hieararchical Bayesian modeling

• take data from all three tasks
• infer latent subjective & "population-level beliefs"
• specify "link functions"
  • how do subjective beliefs generate observable data

dummy

• implemented in JAGS
• 50,000 samples after a burn in of 100,000
• convergence checks: visually and \(\hat{R}\)

• subjective beliefs \(P_{ij}\) may differ from population-level mean (good!)

• BH task averages reasonably approximate mean \(Q_j\) (excellent!)

dummy

• BH task averages are practical and reasonable approximations of the population-level central tendency of individual beliefs

dummy

• Bayesian data analysis is a great tool to specify and test ideas about how unobservables (like beliefs) relate to each other and how they generate observable data
  • inferences based on diverse data from different tasks

• \(w \sim \text{Gamma}(2,0.1)\)
• \(Q_{j} \sim \text{Dirichlet}(1,\dots, 1)\)
• \(P_{ij} \sim \text{Dirichlet}(w Q_j)\)

dummy

• \(\kappa \sim \text{Gamma}(5,5)\)
• \(\sigma \sim \text{Gamma}(0.0001, 0.0001)\)
• \(s_{ijk} \sim \text{logistic}(\text{Norm}(\text{logit}(P_{ijk}), \sigma), \kappa)\)

dummy

• \(a \sim \text{Gamma}(2,1)\)
• \(n_{ij} \sim \text{Categorical}(\text{exp}(a P_{ij}))\)

dummy

qplot(sample(x = 5, size = 1000, replace = TRUE, prob = exp(a *(1:5))))

• \(b \sim \text{Gamma}(2,1)\)
• \(c_{ijl} \sim \text{Bern}(\text{exp}(b \ (p^{\text{high}}_{ijl}, p^{\text{low}}_{ijl}))\)

\[p^\text{high}_{ijl} = \begin{cases} 2 & \text{if $mode(P_{ij})$ is closer to higher} \\ & \text{ bin of $l$ than to lower bin } \\ 1 & \text{if equal distance} \\ 0 & \text{otherwise} \end{cases}\]

\[p^\text{low}_{ijl} = 2 - p^\text{high}_{ijk}\]

example

• what's more likely: John scored 10-12, or John scored 26-28 points?
• subjective belief says: most likely John scored 14-16
• lower bin is closer, so it's more likely to be chosen